Skip to content

ESP32: allocate the largest arrays per session, sizes from the build - #1

Merged
finger563 merged 5 commits into
esp32p4from
esp32/alloc-arrays-at-start
Nov 21, 2026
Merged

finger563 merged 5 commits into
esp32p4from
esp32/alloc-arrays-at-start

Conversation

@finger563

@finger563 finger563 commented Nov 21, 2026 •

Copy link
Copy Markdown

Summary

On the ESP32 the engine's largest arrays are allocated when the game starts and freed when it quits, instead of being static.

In tab5-emu every game core's statics share static PSRAM, so Jedi Knight's 1.3 MB of arrays was reserved while any other game ran. With esp-cpp/tab5-emu#34 (generated globals.c and the port's allocator), static PSRAM goes from 2.20 MB to 1.18 MB. Every other game gets about 995 KB more.

  • PLATFORM_SESSION_ARRAYS is set in engine_config.h for TARGET_ESP32 only. It turns the moved arrays into pointers; other targets keep the static arrays.
  • src/Platform/ESP32/jk_esp_arrays.c allocates one zeroed block at launch (before the engine task starts) and frees it after the engine stops. A relaunch allocates again, zeroed.
    • The block holds sithAI_actors, jkPlayer_playerInfos, jkPlayer_otherThings, jkPlayer_aMotsInfos, aEntryStack, sithCollision_aCollisions, sithSurface_aSurfaces and the port's directory cache, 362 KB at the defaults.
    • Uses of sizeof became explicit counts, and the extern in jkPlayer.h became a pointer.
  • aActiveFaces / aActiveSpans (630 KB) are no longer reserved. This port draws with the z-buffered rasterizer (RDRASTER_SW_ZBUFFER), so rdActive_DrawScene() never runs. They are allocated only if that rasterizer is ever the renderer, and rdActive_AddActiveFace checks the configured count.
  • aBounce in sithSound.c (16 KB) is compiled out on the ESP32. It was for Dreamcast/DSi memory that rejects byte writes.
  • Sizes from the build (#ifndef defaults are today's values):
    • SITHAI_MAX_ACTORS;
    • the directory cache size;
    • the material cache's eviction threshold and age (JK_ESP_MATERIAL_PURGE_FREE_KB, JK_ESP_MATERIAL_PURGE_AGE).
  • Also: jkPlayer_Shutdown's MoTS loop freed jkPlayer_otherThings[i] instead of jkPlayer_aMotsInfos[i].

None of these arrays was the texture cache. Textures are allocated per material on the heap and evicted by the material cache.

Testing (Tab5)

  • Started a new game through the cutscenes into level 1: 9.8 MB of PSRAM free in the level.
  • Played, saved to a new slot and loaded it back in the same session.
  • Quit, relaunched, and played GBA afterwards: 6,420 KB free, against 5,425 KB on main.
  • Known and pre-existing (main does the same): loading a save before the title screen crashed in sithKeyFrame_GetByIdx. Fixed in ESP32: loading a save before the title screen no longer crashes; once-per-process buffers freed with the engine #2, stacked on this PR, which also frees the once-per-process lexer and mixer buffers that fragmented the heap.

🤖 Generated with Claude Code

https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY

finger563 and others added 2 commits November 20, 2026 16:58
…m the build

The firmware runs one game at a time but reserves every core's .bss in PSRAM
all the time, so ~1 MB of this engine's static arrays sat unused while the
other cores ran. With PLATFORM_SESSION_ARRAYS (engine_config.h, TARGET_ESP32
only) they are pointers to buffers the platform allocates zeroed before the
engine starts and frees after it has shut down
(Platform/ESP32/jk_esp_arrays.c, one block from the engine heap):

- sithAI_actors (SITHAI_MAX_ACTORS x 672 B, 84 KB at 128)
- jkPlayer_playerInfos, jkPlayer_otherThings, jkPlayer_aMotsInfos,
  aEntryStack, sithCollision_aCollisions, sithSurface_aSurfaces (the globals
  cog templates emit them as pointers under the define)
- the port's directory-existence cache (jk_esp_fs.c, 64.5 KB at 512)

Not allocated at all: the active-edge rasterizer's pools (aActiveFaces,
aActiveSpans, 616 KB): with RDRASTER_SW_ZBUFFER the world is drawn by
rdZRaster and rdActive_DrawScene() never runs; it now returns while the
pools are NULL. sithSound's 16 KB bounce buffer is compiled out on ESP32,
where no memory is word-addressable-only.

The build may set SITHAI_MAX_ACTORS, RDACTIVE_MAX_FACES, RDACTIVE_MAX_SPANS
and JK_ESP_FS_DIRCACHE (#ifndef defaults are today's values);
rdActive_AddActiveFace checks the configured face count instead of 0x3ff.
sizeof() uses on the moved arrays are now explicit counts. The other targets
keep the static arrays.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
…n MoTS typo

- Window_ESP32.c: the gentle material (texture) cache eviction's free-memory
  threshold (2560 KB) and age (900 rendered frames) are now
  JK_ESP_MATERIAL_PURGE_FREE_KB and JK_ESP_MATERIAL_PURGE_AGE, with #ifndef
  defaults at today's values, so the build can tune them (tab5-emu's Kconfig).
  The cache itself has no fixed size; these are its only knobs. 0 KB turns
  the gentle eviction off (eviction only on allocation failure).
- jkPlayer_Shutdown: the MoTS loop freed jkPlayer_otherThings[i].polylineThing
  (already zeroed just above) instead of jkPlayer_aMotsInfos[i]'s, like the
  playerThings loop does. In practice nothing leaked (rdThing_FreeEntry frees
  only joint arrays and a puppet, which a polyline thing has none of), but it
  now frees the right one.
- jk_esp_arrays.c: comments (the rasterizer pools keep rdActive.h's sizes;
  jk_esp_arrays_bytes() is used by tab5-emu's debug allocation dump).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
finger563 and others added 3 commits November 20, 2026 21:02
…pause-menu load closes the engine menus

A save loaded from the tab5-emu pause menu while the intro was still
playing crashed in sithKeyFrame_GetByIdx (load fault at +0x20,
lw a3,364(a4) with a4 = sithWorld_g_pStaticWorld = NULL; core dump: a4=0,
a3=0xffff7fff, a5=0x22, i.e. static keyframe 0x8022 on the first track of
the first DSS_PUPPETSTATUS). The static world (static.jkl) and the
inventory bins (items.dat) are loaded by jkMain_TitleShow, which runs
after the intro and before any menu can start a level; the pause menu's
load switches VIDEO -> UNK -> GAMEPLAY directly, so the level and the
save's messages were processed without them. Loading in a session that
already had a level (or that had shown the title) never hit this.

- jkMain_GameplayShow: if no static world is open yet, open static.jkl
  and load items.dat, as jkMain_TitleShow and Main_StartupDedicated's
  autostart do. A no-op on every path that went through the title.
- jk_esp_engine_load: before the title the Players dialog has not read a
  profile either (controls have no bindings): read the remembered player's
  profile, or fall back to the default controls.
- jk_esp_engine_load at an engine menu (main menu, Players dialog, Single
  Player, Setup, the in-game Esc menu): the level switch only happens once
  no modal menu is open, and on RETRO targets a menu does not close by
  itself for jkGuiRend_thing_four, so the load waited for the user to leave
  the menus (and New Game replaced it). jkGuiRend_ESP32_bCloseMenus makes
  every menu return -1 and jkGuiMain_Show leave its loop until
  jkMain_GuiAdvance performs the switch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
… / new player screens; static.jkl before the episode GOB; quit at the main menu returns

Review follow-ups to the fresh-load fix (0ac4f63):

- jkGuiRend_ESP32_bCloseMenus made every menu return -1, but the Force
  and single-player tally screens answer -1 with an "abort?" Yes/No (which
  then also returned -1, i.e. No) and reopened, and the new player dialog
  reopens on -1 while no profile exists: a load at Esc > Jedi Powers, the
  end-of-level tally or star screen, the dark/light choice or a first-run
  Players dialog spun forever and Quit could not end it. Those screens now
  return while the flag is set (the new player dialog with the default
  controls, since no profile was read).
- jkMain_EndLevelScreenShow, jkMain_ChoiceShow and the cutscenes menu fall
  back to the main menu on -1, which replaced the pending load (the cart
  reported success): skipped while the flag is set.
- jkMain_VideoLeave starts the next level when an inter-level cutscene
  (VIDEO3 / VIDEO4) ends, and jkMain_GuiAdvance takes that state over the
  one it is switching to: a load during one was replaced by the next level.
  Skipped when switching to JK_GAMEMODE_UNK (only ever a load).
- static.jkl and items.dat for a level reached without the title are now
  opened in jkMain_UnkShow, with the title's loading screen, before
  jkMain_UnkTick mounts the save's episode GOB: jkRes searches the episode
  directory first, so in jkMain_GameplayShow the session's static world
  could pick up files from that episode. GameplayShow keeps the guard as a
  fallback.
- The flag is reset in jkGuiRend_Shutdown, so it cannot reach the next
  launch.
- A quit from outside the engine (Window_MessageLoop) also sets it: only
  cancelling the active menu left jkGuiMain_Show reopening the main menu,
  so a quit at JK's main menu never returned and jk::deinit waited forever.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
…h the engine

Both are made once per process: the lexer's 16 KB input buffer by the first
COG parse (during the first level load) and the software mixer's chunk
buffer by the first stdSound_Startup. In tab5-emu the engine runs in a
process that outlives it, so they stayed in the middle of the PSRAM heap
after the game: the first Jedi Knight session split the largest free block
from 18.4 MB to 9.5-12.5 MB for the rest of the boot.

- sithCogParse_ESP32_ReleaseLexer() deletes the lexer's buffer and resets
  yy_init; yyrestart() makes a new buffer when there is none, as newer
  flex does (this one passed the NULL buffer on).
- stdSound_ESP32_Release() frees the mix buffer and clears stdSound_bInitted
  so the next startup makes a new one.
The port calls both once the engine and its audio task have stopped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant